The present invention relates to an image sensing apparatus capable of determining optimum image sensing conditions for obtaining a three-dimensional image. The present invention also relates to an image processing apparatus which edits a three-dimensional image in accordance with the image sensing conditions.
Conventionally, there is a technique, such as the one published on the Journal of Television Society, Vol. 45, No. 4 (1991), pp. 453-460, to obtain a three-dimensional shape of an object. There are basically two methods, a passive method and an active method, for obtaining the three-dimensional shape of the object, as described in the above article.
A typical method as the passive method is a stereo imaging method which performs triangulation on an object by using two cameras. In this method, corresponding points of a part of an object are searched in both the right and left images, and the position of the object in the three dimensional space is measured on the basis of the difference between the positions of the searched corresponding points in the right and left images.
Further, there are a method using a range finder and a slit projection method as the typical active method. In the former method, distance to the object is obtained by measuring the elapsed time between emitting light toward the object and receiving the light reflected by the object. In the latter method, a three-dimensional shape is measured on the basis of deformation of a shape of a light pattern, whose original pattern is a slit shape, projected on an object.
However, the main purpose of the aforesaid stereo imaging method is to calculate information on distance between fixed positions where the cameras are set and the object, and not to measure the entire object. Consequently, a three-dimensional shape can not be obtained in high precision.
Further, an apparatus adopting the active method is large, since it has to emit a laser beam, for example, to an object the manufacturing cost is, therefore, high.
Generation of a three-dimensional shape of the object on the basis of two-dimensional images requires a plurality of images sensed at a plurality of image sensing points. However, since the object has a three-dimensional shape, image sensing parameters (e.g., depth of focus, angle of view) which are suited to the object and each image sensing point needs to be set for performing image sensing from a plurality of image sensing points.
However, in any of the aforesaid methods, cameras are not controlled flexibly enough to respond to a dynamic image sensing method, such as sensing an object while moving around it.
Therefore, the present invention is aimed at solving the aforesaid problem, i.e., to realize a dynamic image sensing method in which an object is sensed at a plurality of image sensing points around it.
Meanwhile, an image of an object which is seen from an arbitrary viewpoint is sometimes reproduced on a two-dimensional display on the basis of obtained three-dimensional data of the object.
For example, it is possible to input images sensed by an electronic camera into a personal computer, or the like, and edit them. In this case, a scene is divided into a plurality of partial scenes, and then is sensed with an electronic camera. The images corresponding to the plurality of partial scenes is projected with having some overlapping portions. More specifically, the sensed images are inputted into a personal computer, then put together by using an application software so that the overlapping portions are projected overlapping each other. Thereby, it is possible to obtain an image of far wider angle of view than that of an image obtained in a single image sensing operation by the electronic camera.
However, the main purpose of the aforesaid stereo imaging method is to calculate information on distance between a fixed position where the camera is set and the object, and not to measure the three-dimensional shape of the entire object.
Further, since a laser beam is emitted to the object in the active method, it is troublesome to use an apparatus adopting the active method. Furthermore, in any conventional method, cameras are not controlled flexibly enough to respond to a dynamic image sensing method, such as sensing an object while moving around it.
In addition, two view finders are necessary in the conventional passive method using two cameras, and it is also necessary to perform image sensing operation as seeing to compare images on the two view finders, which increases manufacturing cost and provides bad operability. For instance, there are problems in which it takes time to perform framing or it becomes impossible to obtain a three-dimensional shape because of too small of an overlapping area.
Further, an image generally dealt with in an office is often printed out on paper eventually, and types of images to be used may be a natural image and a wire image which represents an object with outlines only. In the conventional methods, however, to display an image of an object faithfully on a two-dimensional display on the basis of three-dimensional shape data of the object is the main interest, thus those methods are not used in offices.
The present invention has been made in consideration of the aforesaid situation, and has as its object to provide an image sensing apparatus capable of placing an object, whose three-dimensional shape is to be generated, under the optimum image sensing conditions upon sensing the object from a plurality of image sensing points without bothering an operator.
It is another object of the present invention to provide an image sensing apparatus capable of setting sensing parameters for an optical system so that an entire object falls within the optimum depth of focus at image sensing points.
A further object of the present invention is to provide an image sensing apparatus which senses an image of the object with the optimum zoom ratio at each of a plurality of image sensing points.
Yet a further object of the present invention is to provide an image sensing apparatus capable of notifying an operator of achievement of the optimum image sensing conditions.
Yet further object of the present invention is to provide an image sensing apparatus capable of storing the optimum image sensing conditions.
Yet a further object of the present invention is to provide an image sensing apparatus which determines whether the optimum image sensing conditions are achieved or not by judging whether there is a predetermined pattern in an image sensing field.
Yet a further object of the present invention is to provide an image sensing apparatus capable of re-sensing an image.
Yet a further object of the present invention is to provide an image sensing apparatus whose operability is greatly improved by informing an operator when he/she is to press a shutter.
Yet a further object of the present invention is to provide an image sensing apparatus capable of determining a displacing speed of a camera upon inputting an image, thereby improving operability as well as quality of an input image.
Yet a further object of the present invention is to provide a single-eye type image sensing apparatus capable of inputting an image of high quality thereby obtaining a three-dimensional shape in high precision and reliability.
Yet a further object of the present invention is to provide an image sensing apparatus capable of always sensing characteristic points to be used for posture detection within a field of view, thereby preventing failing an image sensing operation.
Yet a further object of the present invention is to provide an image processing apparatus capable of generating an image of an object which is seen from an arbitrary viewpoint on the basis of three-dimensional shape information on images sensed at a plurality of image sensing points, and capable of forming a file.
Yet a further object of the present invention is to provide an image processing apparatus capable of generating an image of an object which is seen from an arbitrary viewpoint on the basis of three-dimensional shape information on images sensed at a plurality of image sensing points, and capable of forming a file, and further editing the three-dimensional image by synthesizing the file with other file.
Yet a further object of the present invention is to provide an image processing apparatus which generates a three-dimensional image from images sensed under the optimum image sensing conditions.
Yet a further object of the present invention is to provide an image processing apparatus which converts three-dimensional shape data, obtained based on sensed images, into a two-dimensional image of an object which is seen from an arbitrary viewpoint.
Yet a further object of the present invention is to provide an image processing apparatus which combines a document file and a three-dimensional image.
Yet a further object of the present invention is to provide an image processing apparatus which stores information on background of an object.
Yet a further object of the present invention is to provide an image processing apparatus which combines an image data file with another file, and has a three-dimensionally displaying function.
Yet a further object of the present invention is to provide an image processing apparatus in which three-dimensional shape data of an object is calculated with a software installed in a computer, the three-dimensional shape data of the object is converted into an image of the object seen from an arbitrary viewpoint, and a file of the image data is combined with another file.
Yet a further object of the present invention is to provide an image sensing apparatus capable of detecting overlapping areas in a plurality of images sensed at a plurality of image sensing points.
Yet a further object of the present invention is to provide an image sensing apparatus which displays overlapping portions of images sensed at a plurality of image sensing points in a style different from a style for displaying non-overlapping portions.
Yet a further object of the present invention is to provide an image processing apparatus capable of re-sensing an image.
Other features and advantages of the present invention will be apparent from the following description taken in conjunction with the accompanying drawings, in which like reference characters designate the same or similar parts throughout the figures thereof.